Systems and methods for generating orthotic device models from user-based data capture

ABSTRACT

A method for generating an orthotic device is disclosed. The method includes receiving data from a client device of a patient, the data comprising patient information and image data representative of a body part of the patient. The method further includes generating, based on the image data, three-dimensional model data representative of the body part, and generating parametric CAD model data of the orthotic device based on the three-dimensional model data and the patient information. The parametric CAD model data is transmitted to a three-dimensional printer, wherein the three-dimensional printer is to generate the orthotic device based on the parametric CAD model data

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional patentapplication Ser. No. 15/894,241, filed Dec. 12, 2018, which is acontinuation of U.S. Non-Provisional patent application Ser. No.14/961,769, filed Dec. 7, 2015, which is a continuation of U.S.Non-Provisional patent application Ser. No. 14/677,894, filed Apr. 2,2015, which is a continuation of U.S. Non-Provisional application Ser.No. 14/341,632, filed Jul. 25, 2014, which claims the benefit ofpriority of U.S. Provisional Patent Application No. 61/981,753, filedApr. 19, 2014, and U.S. Provisional Patent Application No. 61/858,685,filed Jul. 26, 2013, all of which are hereby incorporated by referenceherein in their entireties.

TECHNICAL FIELD

This disclosure relates to the field of corrective orthotic devices, inparticular, to generating models from user-captured data to produceorthotic devices.

BACKGROUND

An orthotic insert is a type of orthotic device that, when inserted intoa shoe and applied to a foot, supports the foot by redistributing groundreaction forces while properly aligning foot joints during motion.Orthotic inserts are typically used to treat biomechanical deformitiesas well as inflammatory conditions (e.g., plantar fasciitis) inpatients.

Various methods have been employed to produce orthotic inserts. Forexample, plaster cast, gait scanning, and laser scanning methods attemptto capture plantar geometry in a weight bearing position. However, suchmethods are generally slow in acquiring orthotic data, are expensive,and are limited in the range of characteristics that they can provide tothe resulting orthotic device. In such methods, the orthotic device iscustomizable insofar as it is designed with a particular ailment inmind, while the treatment is implemented as a one-size-fits-all solutionthat may be far from optimal for some patients.

Moreover, current methods of orthotic insert production are generallylimited to the machining of hard materials (top down approaches). Thisalso limits the range of characteristics (flexibility, shock absorption,weight, etc.) of the end product. Shapes of the orthotic inserts tend tobe mixed and matched from a database, which may result in orthoticinserts that are unique to a particular lab or production facility butnot to a particular patient.

SUMMARY

The following is a simplified summary of the disclosure in order toprovide a basic understanding of some aspects of the disclosure. Thissummary is not an extensive overview of the disclosure. It is intendedto neither identify key or critical elements of the disclosure, nordelineate any scope of the particular implementations of the disclosureor any scope of the claims. Its sole purpose is to present some conceptsof the disclosure in a simplified form as a prelude to the more detaileddescription that is presented later.

In one aspect, a method includes receiving, from a client device, datarelated to a patient, the data including patient information and imagedata, in which the image data is representative of a body part of thepatient. The method further includes generating, based on the imagedata, three-dimensional model data representative of the body part. Themethod further includes generating parametric computer-aided design(CAD) model data for an orthotic device based on the three-dimensionalmodel data and the patient information. The method further includestransmitting the parametric CAD model data to a three-dimensionalprinter, wherein the three-dimensional printer is to generate theorthotic device based on the parametric CAD model data.

In another aspect, a method includes receiving a plurality of userinputs at a client device, the user inputs corresponding to patientinformation. The method further includes generating one or moreindicators to guide the capturing of image data of a body part. Themethod further includes capturing the image data of the body part. Themethod further includes transmitting the patient information and imagedata to a remote device for processing, wherein the remote device is togenerate, based on the patient information and the image data,three-dimensional model data representative of the body part.

In one or more of the disclosed implementations, computing devices forperforming the operations of the above described implementations arealso disclosed. Additionally, in implementations of the disclosure, acomputer readable storage media stores methods for performing theoperations of the above described implementations.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings, in which:

FIG. 1A illustrates an example system architecture in accordance with animplementation of the disclosure;

FIG. 1B is a block diagram illustrating features of a model generationcomponent in accordance with an implementation of the disclosure;

FIG. 2A illustrate the capture of image data according to animplementation of the disclosure;

FIG. 2B illustrates the capture of image data according to anotherimplementation of the disclosure;

FIG. 3 illustrates image processing for model generation according to animplementation of the disclosure;

FIG. 4A is an illustration of a density lattice used in generating anorthotic insert according to an implementation of the disclosure;

FIG. 4B shows an orthotic insert produced according to an implementationof the disclosure;

FIG. 5A is a flow diagram illustrating a method for producing anorthotic device according to an implementation of the disclosure;

FIG. 5B is a flow diagram illustrating a method for capturing data froma user for use in producing an orthotic device according to animplementation of the disclosure; and

FIG. 6 is a block diagram illustrating an exemplary computer system foruse in accordance an implementation of the disclosure

DETAILED DESCRIPTION

Implementations are described for producing orthotic devices fromuser-captured data. Image data of a body part of a patient (e.g., thepatient's foot) can be captured using a client device, such as a mobiledevice having a camera. An interface implemented on the client devicecan instruct a user of the device (e.g., the patient, a physician, anassistant, etc.) to capture image data (e.g., images and/or video) ofthe body part. For example, the interface may utilize various indicators(e.g., visual cues) to guide data capture. If the device includes aninertial measurement unit (IMU), then IMU data may also be capturedduring the image data capture, which may facilitate downstream dataprocessing. The device can also allow for data entry of patientinformation, which can include physical parameters related to thepatient (e.g., height, weight, age, pre-existing medical conditions,etc.).

The captured image data, IMU data, and patient information may betransmitted to a server, which uses the received data to generate athree-dimensional (3D) model of the body part. In doing so, relevantmeasurements related to the body part may be extracted, and are in turnused to generate a parametric CAD model of an orthotic device (e.g., anorthotic insert). The parametric CAD model may be transmitted, forexample, to a manufacturing facility that can print the orthotic deviceusing a 3D printer.

The implementations of the present disclosure provide several advantagesover traditional orthotic device technologies, and orthotic inserttechnologies in particular. The implementations described hereineffectively transform a mobile device into a scanner that is portable,lightweight, and accurate, allowing for a patient to perform his/her ownscans without requiring trained medical personnel. In addition, adigital 3D model of a foot in digital format facilitates error checkingof the quality of the scan through visualization and comparison withprior scans and studies. The digital 3D model can continue to beinspected, re-used, or re-processed by the algorithm or updated based onnew algorithms and/or updated patient data. Once complete, the digital3D model can be transmitted from a patient device or a device of thephysician to a manufacturing facility for production of the orthoticinsert, alleviating the need to physically transport a mold or cast(which typically cannot be re-used) to the manufacturing facility. Thelevel of accuracy and consistency in the digital 3D model of a foot, asprovided by the disclosed implementations, outperforms plaster cast andgait scanning methods. Moreover, the implementations described hereincan produce an orthotic insert with variable density by incorporatingsurface and sub-surface structures that further customize the mechanicalproperties of the orthotic insert, resulting in an orthotic insert thatis extremely thin, cushioned, and patient-optimized all at once.

The term “orthotic device”, as used herein, refers to any device worn byor externally applied to an individual that provides neuromuscularsupport to the individual, provides skeletal support to the individual,and/or provides prophylactic functionality. The term “correctivedevice”, as used herein, refers to a type of orthotic device thatprovides a therapeutic benefit to an individual (e.g., who may bereferred to herein as “a patient”) when worn by or externally applied byto the individual. While the implementations herein are described withrespect to orthotic devices for treating or supporting a patient's foot(i.e., orthotic shoe inserts), it is to be understood that the systemsand methods described herein are applicable to the production of othertypes of devices. For example, the implementations described herein aregenerally applicable to the production of devices that are customized tofit to the human body, devices utilizing customization and optimizationrelated to human activity, bio-mechanics, and anatomy, processes thatcan be applied to consumer devices with or without specialized hardwareor skill, and devices that utilize components or structures that wouldbe difficult to produce in mass quantities with traditionalmanufacturing approaches. Such devices may include, but are not limitedto, helmets, body armor, sports equipment, prosthetics, casts, splints,clothing, furniture, vehicle seats, vehicle or robotic controlmechanisms, physical therapy devices, gloves, surgical instruments, andsterile medical packing.

FIG. 1A illustrates an example system architecture 100, in accordancewith an implementation of the disclosure, for generating an orthoticdevice. The system architecture 100 includes client devices 110A-110Z, amodeling server 120, a production server 130, 3D printers 132A-132Z, adata store 140, and a network 150.

In one implementation, the client devices 110A-110Z may each includecomputing devices such as personal computers (PCs), laptops, mobilephones, smart phones, tablet computers, netbook computers etc. Clientdevices 110A-110Z may also be referred to as “user devices”. Anindividual user may be associated with (e.g., own and/or use) one ormore client devices (e.g., one or more of client devices 110A-110Z).Client devices 110A-110Z may each be owned and utilized by differentusers at different locations. As used herein, a “user” may refergenerally to an individual operator of one or more of client devices110A-110Z, and may be a patient for which an orthotic device is to beproduced, a clinician or physician who may be involved in thepreparation of the orthotic device in conjunction with, or on behalf of,the patient, an assistant to the patient, etc.

The client devices 110A-110Z may each implement user interfaces112A-112Z, respectively. Each of user interfaces 112A-112Z may allow auser of the respective client device 110A-110Z to send and receiveinformation to one or more of the modeling server 120 and the productionserver 130. For example, one or more of the user interfaces 112A-112Zmay be a web browser interface that can access, retrieve, present,and/or navigate content (e.g., web pages such as Hyper Text MarkupLanguage (HTML) pages) provided by the modeling server 120. In oneimplementation, one or more of the user interfaces 112A-112Z may be astandalone application (e.g., a mobile app), which may have beenprovided by the modeling server 120 (e.g., as a downloadableapplication), that allows a user of a respective client device 110A-110Zto send and receive information to the modeling server 120. In oneimplementation, the user interfaces 112A-112Z guide their respectiveusers in capturing image data of a body part, which is utilizeddownstream by the modeling server 120 to generate a 3D model of the bodypart. The term “image data” is intended to include any type of visualdata that can be captured by an optical instrument (e.g., a photographiccamera, a charge-coupled device (CCD) camera, an infrared camera, etc.),including videos, static images, and video frames.

In one implementation, one or more of the client devices 110A-110Z maycapture and store image data 114A-114Z, respectively, which may includeone or more static images, videos, and/or audio data (e.g., which may beembedded within the video data or may be a separate audio track). Theimage data 114A-114Z may be made accessible to other devices of thesystem architecture 100 via the network 150. For example, captured imagedata may be transmitted to (e.g., streamed in real-time during captureor transmitted at a later time after capturing the data) the modelingserver 120 and/or the data store 140. Each of client devices 110A-110Zmay also capture IMU data 116A-116Z, respectively, which may includegyroscopic data, magnetometer data, GPS data, etc., captured by therespective client device while in use. For example, IMU data 116Acaptured while a user is operating the client device 110A to captureimage data 114A of his/her foot may be used to estimate the orientationof the client device 110A (e.g., if the client device is a mobile devicewith a camera), and may later be leveraged to identify frames of videothat best capture the foot to facilitate downstream data processing. TheIMU data 116A-116Z may be made accessible to other devices of the systemarchitecture 100 via the network 150.

In one implementation, the modeling server 120 may be one or morecomputing devices (such as a rackmount server, a router computer, aserver computer, a personal computer, a mainframe computer, a laptopcomputer, a tablet computer, a desktop computer, etc.), data stores(e.g., hard disks, memories, databases), networks, software components,and/or hardware components. The modeling server 120 may include a modelgeneration component 160 (which may be executed by a processing deviceof the modeling server 120) that is capable of generatingthree-dimensional (3D) models of a patient's body part (e.g., thepatient's foot) based on image data-captured by the patient/user of oneof client devices 110A-110Z, as well as additional patient-related data(e.g., medical data). The model generation component 160 may also becapable of generating a parametric CAD model of an orthotic device basedon the 3D model of the patient's body part. In some implementations, themodel generation component 160 may be implemented on a different devicethan modeling server 120. For example, in some implementations, one ormore of client devices 110A-110Z may implement the model generationcomponent 160, and modeling server 120 may be omitted from the systemarchitecture 100. In other implementations, the modeling server 120 maybe combined with the production server 130 as a single server. In oneimplementation, the modeling server 120 may utilize high performancecomputing resources (e.g., available via the Internet) by outsourcingdata processing functions to high-performance computing devices.

In one implementation, the production server 130 may be one or morecomputing devices (such as a rackmount server, a router computer, aserver computer, a personal computer, a mainframe computer, a laptopcomputer, a tablet computer, a desktop computer, etc.), data stores(e.g., hard disks, memories, databases), networks, software components,and/or hardware components. The production server 130 may receiveparametric CAD models data from the modeling server 120 via the network150. The parametric CAD model data may be converted, by the productionserver 130, into a format suitable for 3D printing prior to transmittingthe data to one or more of the 3D printers 132A-132Z.

In one implementation, the 3D printers 132A-132Z are communicativelycoupled to the production server 130, as illustrated. In someimplementations, one or more of the 3D printers 132A-132Z may be coupledto one or more of the devices of the system architecture 100 in additionto the production server 130, which may be communicatively coupled tothese devices via the network 150. Each of the 3D printers 132A-132Z maybe capable of one or more of fused deposition modeling,stereolithography, selective laser sintering, or any type of 3D printingtechnology as would be understood by one of ordinary skill in the art.

In one implementation, the data store 140 may be a memory (e.g., randomaccess memory), a cache, a drive (e.g., a hard drive), a flash drive, adatabase system, or another type of component or device capable ofstoring data. The data store 140 may also include multiple storagecomponents (e.g., multiple drives or multiple databases) that may alsospan multiple computing devices (e.g., multiple server computers), andmay be cloud-based. In some implementations, the data store 140 may be apart of the modeling server 120. In some implementations, the data store140 may be distributed among and accessible to one or more of the clientdevices 110A-110Z, the modeling server 120, and the production server130. One or more of the devices of the system architecture 100 mayutilize the data store 140 to store public and private data, and thedata store 140 may be configured to provide secure storage for privatedata (e.g., patient-specific information).

In one implementation, the network 150 may include a public network(e.g., the Internet), a private network (e.g., a local area network(LAN) or wide area network (WAN)), a wired network (e.g., Ethernetnetwork), a wireless network (e.g., an 802.11 network or a Wi-Finetwork), a cellular network (e.g., a Long Term Evolution (LTE)network), routers, hubs, switches, server computers, or a combinationthereof. In some implementations, the network 150 may be a combinationof different types of networks. Image data 114A-114Z and IMU data116A-116Z of any of client devices 110A-110Z may be transmitted tomodeling server 120 and/or production server 130 via the network 150.Likewise, 3D model data and parametric CAD model data may be transmittedfrom the modeling server 120 to any one of the client devices 110A-110Zand the production server 130 via the network 150.

FIG. 1B is a block diagram illustrating features of a model generationcomponent 160 in accordance with an implementation of the disclosure.The model generation component 160 may be the same as its identicallynamed counterpart of FIG. 1A. In one implementation, the modelgeneration component 160 includes a client interface module 162, animage processing module 164, a 3D model generation module 166, and a CADmodel generation module 168. More or less components may be included inthe model generation component 160 without loss of generality. Forexample, two or more of the modules may be combined into a singlemodule, or one of the modules may be divided into two or more modules.In one implementation, one or more of the modules may reside ondifferent computing devices (e.g., different server computers, on aclient device, distributed among multiple client devices, etc.). Themodel generation component 160 was described as being implemented by themodeling server 120 of FIG. 1A, but may be implemented by any of theclient devices 110A-110Z and the production server 130. For example, aclient device (e.g., client device 110A) may be programmed to performsome or all of the functions of the model generation component 160. Whenthe model generation component 160 is implemented on a client device,any functions described with respect to the model generation component160 that “receive”, “transmit”, “generate”, “retrieve”, “identify”,“determine”, “select”, etc., are understood to refer to functionsperformed by sub-systems or sub-modules within the client device ratherthan across a network (e.g., the network 150), as would be appreciatedby one of ordinary skill in the art.

In one implementation, the model generation component 160 iscommunicatively coupled to the data store 140. For example, the modelgeneration component 160 may be coupled to the data store 140 via anetwork (e.g., via network 150). As described with respect to FIG. 1A,the data store 140 may be a memory (e.g., a random access memory), acache, a drive (e.g., a hard drive), a flash drive, a database system,or another type of component or device capable of storing data. The datastore 106 may also include multiple storage components (e.g., multipledrives or multiple databases) that may also span multiple computingdevices (e.g., multiple server computers), and may be cloud-based. Inone implementation, the data store 140 may include patient information142, patient image data 144, 3D model data 146, and CAD model data 148.While data store 140 is illustrated and described with respect to asingle patient, and it is to be understood that data store 140 may storedata associated with multiple patients, and the implementationsdescribed herein may be performed for multiple patients concurrently.

The term “patient information”, as used herein, refers to anyalphanumeric data that may describe one or more physical aspects of apatient. The patient information 142 may include, but is not limited to,the patient's height, weight, age, ethnicity, a pre-existing medicalcondition (e.g., a podiatric medical condition), a measured foot length,shoe size, etc. In some implementations, the patient information isprovided by the patient (e.g., using one of the client devices110A-110Z). For example, prior to being prompted to capture image dataor after capturing image data, the user may be provided with a userinterface, such as a Tillable-form interface, that allows the user toenter physiological and/or medical data associated with the patient(e.g., the user may be the patient, a physician of the patient, or anassistant of the patient or physician). The data may then be transmittedto the modeling server (e.g., modeling server 120) for processing. Insome implementations, the patient information may be provided by aphysician (e.g., using one of client devices 110A-110Z and/or usingmodeling server 120). In some implementations, both the patient and thephysician may provide portions of the patient information. In someimplementations, some or all of the information may have been previouslystored in the data store. In some implementations, the patientinformation may be provided by more than one client device (e.g.,provided by more than one of client devices 110A-110Z).

In one implementation, the patient image data 144 includes images and/orvideo captured by a user of a client device (e.g., one or more of clientdevices 110A-110Z). The patient image data 144 may also include IMU datastored along with the images and/or video (and may be time-synchronizedwith video frames), which may be used by the model generation component160 during image processing.

In one implementation, the model generation component 160 utilizes theclient interface module 162 to send/receive information to/from a clientdevice. The client interface module 162 may provide an interface forrequesting information from the client device. At the user end, a userinterface (e.g., user interface 112A) may be in the form of a web pageor a standalone application that may provide an interface to enterpatient information, as well as instructions to the user as to how tocapture the image data related to a body part of the patient.

In one implementation, as illustrated in FIG. 2A, a user may orient aclient device 204 (which may correspond to any of client devices110A-110Z) to capture images and/or video of a foot 202 of the patientbased in accordance with instructions/indicators provided on a display208 of the client device 204 (e.g., a visual cue). In oneimplementation, the instructions/indicators may alternatively oradditionally include audio cues and/or haptic feedback (e.g., avibration to indicate proper orientation of the client device 204 withrespect to the foot 202). A relative coordinate axis 210 may be definedwhich may serve as a reference point for captured IMU data. As anillustrative implementation, a user captures video of the foot 202 atdifferent orientations with respect to the client device 204 and takestwo or more still images of the weight bearing heel and arch. The videoand/or images may be captured using a built-in camera 206 of the clientdevice 204.

As illustrated in FIG. 2B, indicators (e.g., visual cues) may begenerated for display within an image window 212 within display 208 asthe client device 204 is capturing image data of the foot 202. Theindicators, such as reticles 214, may serve as indicators for guidingthe user to capture relevant regions of the foot in weight-bearing andnon-weight-bearing positions. In one implementation, indicators mayinclude overlaid grids, horizontal lines, vertical lines, andfoot-shaped targets/overlays. As the user orients the built-in camera206 and captures image data, the client device 204 may provideadditional indicators, such as instructions 216, to inform the user ofhis/her progress and next steps. In one implementation, visual, audio,and/or haptic cues may indicate successful progress or problems with thecapture process. For example, IMU data may be used by the client device204 to determine that the camera motion has exceeded a thresholdtranslational or angular speed and/or acceleration, and a warningindication may be generated indicating the data captured is potentiallyunreliable and/or that the data should be recaptured. In someimplementations, visual indicators may change shape as video is capturedin order to guide the user through particularly helpful views of thefoot 202. For example, such visual indicators may include arrows thatdirect the user. Examples of successful views may be generated fordisplay as a result of successful capture or to provide suggestedorientations to the user. For example, side view 218 and back view 220may be generated for display as examples. Side view 218 and back view220 may also correspond to a 3D model of the foot generated by the modelgeneration component 160, which may have computed model data andrelevant measurements 222 and transmitted the relevant data back to theclient device for display. In one implementation, the client device maygenerate a 3D model of the foot in real-time (e.g., when the modelgeneration component 160 is implemented on the client device).

Referring back to FIG. 1B, in one implementation, the model generationcomponent 160 utilizes the image processing module 164. Individualframes may be selected from the video based on image quality (e.g.focus) and IMU data (if available) in order to best represent themultitude of viewing angles contained in the video. In oneimplementation, relative position, translational speed, and angularspeed of a camera of the client device captured by the IMU of the clientdevice. The data captured by the IMU may be used by the image processingmodule 164 to determine the uniqueness of information from a given videoframe from other video frames based on respective positions/orientationsof the other frames, which may be used to save processing time byeliminating video frames that contain redundant data. In addition, thespeed of the client device may serve as an indicator of when motionartifacts (e.g., motion blur from rolling shutter cameras) are likely tobe present, thus allowing frames captured during periods of fast cameramovement to be eliminated.

In one implementation, once the frames are selected, each of theselected frames are filtered to remove background objects based on, forexample, colors appearing in the frame, leaving the foot behind in theimages, as illustrated in FIG. 3. In some implementations, a specialcolored and/or patterned sock may be worn by the patient during imagedata capture to isolate the foot from background objects in the videoframes and still images. The pattern may be designed to aid in pointfinding, matching, and 3D reconstruction. Example patterns include arandomized set of symbols in size, orientation, and color. In general,the pattern may be chosen such that its visual detail is robust againstaberrations that may occur due to the camera/lens components andlighting conditions. In some implementations, 3D reconstruction may beperformed with foot images without an article worn by the patient (e.g.,based on skin tone, sub-surface scattering effects, etc.).

In one implementation, the model generation component 160 utilizes the3D model generation module 166 to generate 3D model data describing thebody part (e.g., a 3D reconstruction of the body part). In oneimplementation, the frames selected by the image processing module 164are used as inputs into an algorithm that performs registration betweeneach frame using unique “key-points”, or identifiable groups of pixelsin the images. The algorithm may be a multi-view 3D reconstructionalgorithm, a structure-from-motion algorithm, or another suitablealgorithm. The relative location of the camera for each frame and theidentified key-points can be combined to identify the locations of thekey-points in 3D space as a point cloud representing an outer surface ofthe foot. From this approach, the plantar surface geometry can beobtained. In some implementations, a similar approach may be applied toother body parts for the purposes of obtaining 3D geometry, including,but not limited to other orthotic devices, prosthetics, and organs. Insome implementations, the approach may be applied to inanimate objects(e.g., medical tools) rather than to body parts.

For certain orientations of the foot, the 3D point cloud datarepresenting the surface of the foot, normal vectors of the points, andthe IMU data combined with successive calculations and transformationsof a 3D object-oriented bounding box may be used. The point cloud modelmay also be converted into a “solid” model of the foot for visualizationpurposes. The 3D model provides the capability to visualize the foot insituations, for example, when it is difficult or impractical to obtain aphysical model of the foot.

In one implementation, the 3D model generation module 166 and the imageprocessing module 164 may derive various measurements from the 3D modeland selected images. The measurements include, but are not limited to,foot length, weight bearing arch height (e.g., maximum weight bearingheight of the plantar arch at the arch location, weight bearing subtalarjoint angle, total adjusted arch height, arch location (e.g., maxplantar arch height location in sagittal and transverse planes in aneutral pose), arch width (e.g., midfoot width, width at peak arch),arch angle, prescription arch height, ball location (e.g., sesamoidlocation along the sagittal plane), mid heel location (e.g., midcalcaneus location along the sagittal plane), ball width (e.g., maximumforefoot width), and heel width (e.g., maximum plantar calcaneus footwidth). The selected images used to perform these derivations mayinclude, but are not limited to, an image of a heel of the foot in thefrontal plane, and an image of a medial arch of the foot in the sagittalplane under weight bearing conditions.

In one implementation, one or more of measured foot length (which mayhave been physically measured and entered as part of the patientinformation), IMU data, or reference symbols on the sock (e.g., patternsor objects attached to a sock), or objects located in the environment ofthe foot may be used to scale the data appropriately to ensure theaccuracy of the derived measurements. In one implementation, a knownshoe size of the patient may be used to determine the foot length. Forexample, foot length, L_(M) and L_(F), for male feet and female feet,respectively, may be determined by:

L _(M)=0.82387415 *S _(M)+18.7012954   Eq. 1

L _(F)=0.83529411 *S _(F)+17.5176470   Eq. 2

where S_(M) and S_(F) are male and female shoe sizes, respectively.

In some implementations, additional parameters may be defined accordingto:

$\begin{matrix}{\mspace{79mu} {{{{arch}\mspace{14mu} {angle}} = {\tan^{- 1}( \frac{{arch}\mspace{14mu} {height}}{{arch}\mspace{14mu} {width}} )}},}} & {{Eq}.\mspace{14mu} 3} \\{{{prescription}\mspace{14mu} {arch}\mspace{14mu} {height}} = {{arch}\mspace{14mu} {width}*{{\tan ( {{{arch}\mspace{14mu} {angel}} + {{subtalar}\mspace{14mu} {angle}}} )}.}}} & {{Eq}.\mspace{14mu} 4}\end{matrix}$

In one implementation, the maximum arch height may be defined to accountfor limitations imposed by the maximum angle that can be made betweenthe arch length and the arch height. The max angle may be determined bythe patient's forefront flexibility, according to:

$\begin{matrix}{{{{maximum}\mspace{14mu} {arch}\mspace{14mu} {height}} = {\frac{{arch}\mspace{14mu} {length}}{2}*{\tan ( {0.45 + \alpha} )}}},} & {{Eq}.\mspace{14mu} 5}\end{matrix}$

where α=0 for a rigid foot (maximum angle is 0.45 radians), α=0.05 for asemi-rigid foot (maximum angle is 0.50 radians), and α=0.1 for aflexible foot (maximum angle is 0.55 radians). In one implementation, ifthe prescription arch height is greater than the maximum arch height,the arch height will be modeled as the maximum arch height. Otherwise,the arch height will be modeled as the prescription arch height. In oneimplementation, if an original arch height is greater than the modeledarch height, the modeled arch height may be set as the original archheight so as to ensure that the orthotic device closely follows thecontours of the foot.

In one implementation, ball width (in millimeters) is determinedaccording to:

narrow ball width=0.2224(Foot Length)+1.0567,   Eq. 6

medium ball width=0.2335(Foot Length)+1.1189,   Eq. 7

wide ball width=0.2380(Foot Length)+1.1403,   Eq. 8

for males, and:

narrow ball width=0.2707(Foot Length)+5.7137,   Eq. 9

medium ball width=0.2853(Foot Length)+6.0209,   Eq. 10

wide ball width=0.2911(Foot Length)+6.1437,   Eq. 11

for females. In some implementations, the ball width may be limitedbased on a particular shoe for which the orthotic insert is designed tofit.

In one implementation, heel width (in centimeters) is determinedaccording to:

heel width=0.1473(Foot Length)+2.3882.   Eq. 12

In one implementation, arch width may be limited according to:

heel width≤arch width≤ball width,   Eq. 13

where the arch width is constrained by these limits after all otherconstraints are applied to the arch width.

In one implementation, arch thickness may be determined based on a rangeof the patient's weight. For example, if the patient's weight in poundsis less than 140 pounds, then the arch thickness is modeled as 2millimeters. If the patient's weight is greater than or equal to 140pounds and less than 180 pounds, the arch thickness is modeled as 3millimeters. If the patient's weight is greater than or equal to 180pounds, the arch thickness is modeled as 4 millimeters.

In one implementation, the model generation component 160 utilizes theCAD model generation module 168 to generate parametric CAD model datafor the orthotic device. The CAD model generation module 168 may usevarious algorithms to determine optimal shapes and mechanical propertiesof the orthotic device, which are described by the parametric CAD modeldata. While the parametric CAD model data may be generated based on the3D model data representing the foot, the data may describe an orthoticinsert that is designed to biomechanically adjust and support the footrather than match the surfaces of the foot from the 3D model. Theparametric CAD model data may include information such as, but notlimited to, lattice designs and layouts, mesh perforation properties(e.g., hole size/shapes, pitch, etc.), curve locations, variations inlattice density, variable thicknesses, etc. In one implementation, the3D model data and patient information (e.g., age, height, weight,medical diagnosis, etc.) may be used as inputs to a parametric CAD modelof the SOL in order to output a customized model in a format suitablefor 3D printing on a 3D printer. It is noted that 3D printing isillustrative, and any suitable manufacturing method may be utilized toproduce the orthotic device.

In one implementation, the CAD model generation module 168 generates aseries of surfaces that may be combined to form top, bottom, andinterior surfaces of a solid model of the orthotic insert, as describedby the parametric CAD model data. In addition, lattice and meshstructures can be generated to define the shape and density profile ofthe orthotic insert, which can be placed within target treatment zones(based on the derived measurements and patient information). Asillustrated in FIG. 4A, a heel portion 402 of the parametric CAD modeldata is depicted, including an outer boundary 404, a treatment boundary406, and density profile 408 (represented as an array of circles thatmay correspond to perforations in the final orthotic insert). Thetreatment boundary 406 and the outer boundary 404 may define a treatmentzone specific for the patient's heel. The density profile 408 may beapplied to the treatment zone to provide greater materialdensity/stiffness at the left side of the heel than the right side. Amapped heel portion 410 shows a result of the mapping after it has beenapplied to the treatment zone. An example of a 3D printed orthoticinsert is shown in FIG. 4B, which is labeled to show relevantmeasurements taken into account during generation of the parametric CADmodel data.

In one implementation, a heel stiffness is defined as:

k=32.6d _(diam)−6.4d _(space)+95.5d _(shell)−13.4,   Eq. 13

where k is the stiffness (in kN/m),d_(diam) is a perforation diameter (in mm),d_(space) is a spacing between perforations (in mm), andd_(shell) is a shell size (in mm).

It is noted that the preceding equations are illustrative examples ofcalculations used to identify measurements for producing an orthoticinsert. Other equations may also be used, and the presentimplementations are not limited to the preceding examples.

FIG. 5A is a flow diagram illustrating a method 500 for producing anorthotic device according to an implementation of the disclosure. Themethod 500 may be performed by processing logic that includes hardware(e.g., circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions run on a processing device to performhardware simulation), or a combination thereof. In one implementation,method 500 may be performed by the model generation component 160 asdescribed with respect to FIGS. 1A and 1B.

Referring to FIG. 5A, method 500 begins at block 505 when data relatedto a patient is received from a client device (e.g., client device110A). The data includes patient information and/or image datarepresentative of a body part of the patient. In one implementation thedata includes IMU data (e.g., IMU data 116A). In one implementation, thebody part is a foot of the patient. In one implementation, the imagedata is representative of a mold of a body part of the patient. In analternative implementation, the image data is representative of aninanimate object. In one implementation, at least some of the data isreceived from an additional client device (such as a client device of aphysician) or data store (e.g., data store 140).

At block 510, 3D model data representative of the body part is generatedbased on the image data. In one implementation, the 3D model data may begenerated based on the image data, the patient information, and/or theIMU data. In one implementation, if the image data includes video, aplurality of frames are extracted from the video based on the IMU data.The 3D model data of the body part may be generated at least in partfrom the plurality of frames.

In one implementation, a unique visual pattern may be identified withina captured image or frame of the image data. The unique visual patternmay be a pattern of symbols, colors, etc., which, if recognized (e.g.,by an image processing algorithm), may be used to extract depthinformation from the captured image or frame. Accordingly, the depthinformation, combined with depth information from different views of thebody part, may be used to generate the 3D model data. In oneimplementation, the unique visual pattern is present on an article wornon the body part of the patient (e.g., a patterned sock as illustratedin FIGS. 2B and 3). In one implementation, the user device may havecaptured depth information (e.g., using an infrared depth sensor), whichmay have been received along with the image data. The depth informationcaptured by the user device may also be used to generate the 3D modeldata.

At block 515, parametric CAD model data of the orthotic device isgenerated based on the 3D model data and the patient information. Theparametric CAD model data may correspond to a 3D representation of theorthotic device. In one implementation, measurements are extracted fromthe 3D model data (e.g., toe, inner arch, outer arch, heel, etc.), whichare used to define parameters for the parametric CAD model data. In oneimplementation, the parameters may define a material density profile forthe orthotic device (e.g., to provide a 3D location-dependent stiffnesswithin the orthotic device). The parametric CAD model data may defineinclude discrete structural features that map to the density profile(e.g., which may be a continuous density profile). The structuralfeatures may include holes, apertures, internal pockets, internalbridges/trusses, etc. In one implementation, if an orthotic device is tohave a stiffness gradient in a top surface that increases from a firstend to a second end, the parametric CAD model may include a plurality ofholes defined in the top surface, with a decreasing number of holes fromthe first end to the second end.

At block 520, the parametric CAD model data is transmitted to a 3Dprinter (e.g., 3D printer 132A). The 3D printer is to generate theorthotic device based on the parametric CAD model data. In oneimplementation, the parametric CAD model data is in a suitable formatfor 3D printing. In one implementation, the parametric CAD model data isconverted into a suitable format for 3D printing prior to transmittingthe parametric CAD model data to the 3D printer. In one implementation,the parametric CAD model data may define one or more layers within theorthotic device. The layers may then be printed and laminated together.In one implementation, the material of the one or more layers mayinclude, but is not limited to, nylon, antimicrobial nylon, ABS plastic,PLA plastic, polyurethane, leather, foam, or combinations thereof. Theprinted orthotic device may be colored by dyeing or through chemical orphoto exposure. The printed orthotic device may be covered with amaterial, such as leather, neoprene, a fabric, etc. In oneimplementation, information on user preferences (e.g., materials,colors, etc.) for the orthotic device may have been received in additionto the patient information and image data. In one implementation,identifying information is 3D printed onto or within the orthoticdevice. For example, a patient's name, a physician's name, a date ofmanufacture, a name of the manufacturing facility, a company name, etc.may be printed onto or within the orthotic device.

FIG. 5B is a flow diagram illustrating a method 550 for capturing datafrom a user for use in producing an orthotic device according to animplementation of the disclosure. The method 550 may be performed byprocessing logic that includes hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (e.g.,instructions run on a processing device to perform hardware simulation),or a combination thereof In one implementation, method 550 may beperformed by a user interface of a client device (e.g., any of userinterfaces 112A-112Z of client devices 110A-110Z, respectively) asdescribed with respect to in FIGS. 1A, 1B, 2A, and 2B.

Referring to FIG. 5B, method 550 begins at block 555 when a plurality ofuser inputs are received at a client device (e.g., client device 110A,which may be a mobile device). In one implementation, the client deviceprovides an interface (e.g., user interface 112A) that prompts the userto enter patient information (e.g., height, weight, age, pre-existingmedical conditions, etc.). In some implementations, the client deviceprompts the user for a subset of patient information (e.g., if somepatient information was received previously). In some implementations,the client device does not prompt the user for patient information, andblock 555 is omitted entirely.

At block 560, one or more indicators are generated at the client deviceto guide capturing of image data of a body part. In one implementation,the one or more indicators comprise at least one of an audio cue (e.g.,audio instructions, a bell indicating successful capture, etc.), avisual cue (e.g., a targeting reticle, a shape of a foot, instructionsin the form of text, etc.), or a haptic cue (e.g., a vibrationindicating successful capture). In one implementation, the client deviceis a mobile device (e.g., such as a tablet, smartphone, etc.) equippedwith a camera such that the user can view a display while capturingimage data. In another implementation, a separate camera may be used tocapture the image data and a separate device may provide indicators tothe user. In some implementations, the user may use a camera to capturethe image data without any feedback indicators (e.g., visual, audio, andhaptic cues).

At block 565, the image data of the body part is captured. In oneimplementation, the patient may be wearing an article (e.g., a sock)having patterns from which depth information can be extracted duringdownstream image processing. In one implementation, after the image datais captured, block 560 and 565 may be repeated to continue capturingimage data (e.g., to acquire additional views of the body part).

At block 570, the patient information and the image data are transmittedto a remote device for processing (e.g., via the network 150). Theremote device (e.g., modeling server 120) is to generate, based on thepatient information and the image data, a first dataset corresponding toa first 3D representation of the body part (e.g., using model generationcomponent 160). In one implementation, at least a portion of the data isprocessed locally (e.g., by the client device). In one implementation,depth data may be transmitted to the remote device, the depth datahaving been captured by the client device during image data capture.

In one implementation, the client device may receive (e.g., from themodeling server 120) the first dataset (e.g., after the first datasetwas generated by the remote device), and generate for display the first3D representation of the body part based on the first dataset. In oneimplementation, the client device may receive the first dataset inresponse to completing a minimum number of scans sufficient forgenerating the first dataset. In one implementation, the client devicemay receive a second dataset corresponding to a second 3D representationof the orthotic device. The second dataset may be, or may be derivedfrom, parametric CAD model data generated based on the first dataset(e.g., by the modeling server 120). The client device may receive thesecond dataset after the second dataset was generated by the remotedevice. The client device may generate for display the second 3Drepresentation of the orthotic device. In one implementation, ifadditional user inputs are received at the client device (e.g., updatedpatient information), the client device may generate for display anupdated 3D representation of the orthotic device based at least in parton the updated patient information. In one implementation, the updatedinformation is transmitted to the remote device, which in turn generatesan updated data set corresponding to the updated 3D representation ofthe orthotic device, and transmits the updated data set to the clientdevice. In one implementation, the client device generates an updateddataset corresponding to the updated 3D representation of the orthoticdevice.

For simplicity of explanation, the methods of this disclosure aredepicted and described as a series of acts. However, acts in accordancewith this disclosure can occur in various orders and/or concurrently,and with other acts not presented and described herein. Furthermore, notall illustrated acts may be required to implement the methods inaccordance with the disclosed subject matter. In addition, those skilledin the art will understand and appreciate that the methods couldalternatively be represented as a series of interrelated states via astate diagram or events. Additionally, it should be appreciated that themethods disclosed in this specification are capable of being stored onan article of manufacture, such as a computer-readable device or storagemedia, to facilitate transporting and transferring such methods tocomputing devices. Accordingly, the term “article of manufacture”, asused herein, is intended to include a computer program accessible fromany computer-readable device or storage media.

FIG. 6 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system 600 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeimplementations, the machine may be connected (e.g., networked) to othermachines in a LAN, an intranet, an extranet, or the Internet. Themachine may operate in the capacity of a server or a client machine inclient-server network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be apersonal computer (PC), a tablet PC, a set-top box (STB), a PersonalDigital Assistant (PDA), a cellular telephone, a web appliance, aserver, a network router, switch or bridge, or any machine capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that machine. Further, while only a singlemachine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein. Some or all of the components of thecomputer system 600 may be utilized by or illustrative of any of clientdevices 110A-110Z, modeling server 120, production server 130, 3Dprinters 132A-132Z, and data store 140.

The exemplary computer system 600 includes a processing device(processor) 602, a main memory 604 (e.g., read-only memory (ROM), flashmemory, dynamic random access memory (DRAM) such as synchronous DRAM(SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 606 (e.g., flashmemory, static random access memory (SRAM), etc.), and a data storagedevice 618, which communicate with each other via a bus 608.

Processor 602 represents one or more general-purpose processing devicessuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processor 602 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,or a processor implementing other instruction sets or processorsimplementing a combination of instruction sets. The processor 602 mayalso be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processor 602 is configured to execute instructions 626for performing the operations and steps discussed herein.

The computer system 600 may further include a network interface device622. The computer system 600 also may include a video display unit 610(e.g., a liquid crystal display (LCD), a cathode ray tube (CRT), or atouch screen), an alphanumeric input device 612 (e.g., a keyboard), acursor control device 614 (e.g., a mouse), and a signal generationdevice 620 (e.g., a speaker). In some implementations, the signalgeneration device 620 may include a vibrational actuator (e.g., forproviding haptic feedback).

The data storage device 618 may include a computer-readable storagemedium 624 on which is stored one or more sets of instructions 626(e.g., software) embodying any one or more of the methodologies orfunctions described herein. The instructions 626 may also reside,completely or at least partially, within the main memory 604 and/orwithin the processor 602 during execution thereof by the computer system600, the main memory 604 and the processor 602 also constitutingcomputer-readable storage media. The instructions 626 may further betransmitted or received over a network 674 (e.g., the network 150) viathe network interface device 622.

In one implementation, the instructions 626 include instructions for oneor more model generation components 160, which may correspond to theidentically-named counterpart described with respect to FIGS. 1A and 1B.While the computer-readable storage medium 624 is shown in an exemplaryimplementation to be a single medium, the terms “computer-readablestorage medium” or “machine-readable storage medium” should be taken toinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more sets of instructions. The terms “computer-readablestorage medium” or “machine-readable storage medium” shall also be takento include any transitory or non-transitory medium that is capable ofstoring, encoding or carrying a set of instructions for execution by themachine and that cause the machine to perform any one or more of themethodologies of the present disclosure. The terms “computer-readablestorage medium” or “machine-readable storage medium” shall accordinglybe taken to include, but not be limited to, solid-state memories,optical media, and magnetic media.

In the foregoing description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that the present disclosure may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the present disclosure.

Some portions of the detailed description may have been presented interms of algorithms and symbolic representations of operations on databits within a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is herein, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the foregoing discussion,it is appreciated that throughout the description, discussions utilizingterms such as “sending”, “receiving”, “transmitting”, “forwarding”,“caching”, “causing”, “providing”, “generating”, “adding”,“subtracting”, “removing”, “analyzing”, “determining”, “enabling”,“identifying”, “modifying” or the like, refer to the actions andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The disclosure also relates to an apparatus, device, or system forperforming the operations herein. This apparatus, device, or system maybe specially constructed for the required purposes, or it may include ageneral purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer- or machine-readable storage medium, such as, butnot limited to, any type of disk including floppy disks, optical disks,compact disk read-only memories (CD-ROMs), and magnetic-optical disks,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example” or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Reference throughout this specification to “an implementation” or“one implementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation. Thus, the appearances of thephrase “an implementation” or “one implementation” in various placesthroughout this specification are not necessarily all referring to thesame implementation. Moreover, it is noted that the “A-Z” notation usedin reference to certain elements of the drawings is not intended to belimiting to a particular number of elements. Thus, “A-Z” is to beconstrued as having one or more of the element present in a particularimplementation.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the disclosure should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

What is claimed is:
 1. A method comprising: receiving, from a clientdevice, data related to a patient, the data comprising patientinformation and image data, wherein the image data is representative ofa body part of the patient; generating, based on the image data,three-dimensional model data representative of the body part; generatingparametric CAD model data for an orthotic device based on thethree-dimensional model data and the patient information; andtransmitting the parametric CAD model data to a three-dimensionalprinter, wherein the three-dimensional printer is to generate theorthotic device based on the parametric CAD model data.